Evidence so far




Research Questions



  1. Do participants read informed consent forms?

  2. Does the information on sharing data influence the participants’ willingness to participate?
    moderated my data type?

  3. Does the option to clarify concerns about the informed consent make a difference in the willingness to participate?

Study 1




Do participants read
informed consent forms?

Study 1

Design

  • Re-use of log-data from past online surveys, that…
    • measured dwell time on the consent page,
    • allow re-use of their data according to the informed consent

  • Sample
    • [Teachers, students, …]
    • \(N_{participants} = ...\) participants
    • from \(N_{studies} = ...\) online surveys ([year begin - year end])

Study 1

Measures

Dwell time on consent: Reading speed (as words per minute):

  • 238: average (silent) reading speed (English) (Brysbaert, 2019)
  • > 500-600: loss of information/comprehension (skimming)
  • > 900: information falls out of working memory before being comprehended (Masson, 1983)

Additional coding

  • Which population? [students, teachers, public, researchers, …]
  • Participants incentivized? How? [none, money, …]
  • Topic of study? [sensitive, somewhat sensitive, not sensitive]

Study 1

Statistical analysis

Study 2a

Conceptual Replication (and expansion) of Study 1



Do participants read
informed consent forms?

replication part
expansion part

Study 2a

Conceptual Replication (and expansion) of Study 1

  • Procedure: Presenting the actual consent, then if participants agree…

  • Measures:
    Dwell time
    Self-reported reading engagement
    Understanding (e.g., via multiple-choice)

  • Controls
    knowledge about typical process of research data handling





“Will the data be shared after the project is completed?”
No, all data will be deleted.
Yes, data will be shared with researchers.
Yes, data will be shared with the public.
Yes, data will be shared via the University’s secured website.
Yes, data will be shared via a trustworthy archive.

Study 2a

Conceptual Replication (and expansion) of Study 1

Study 2a

Conceptual Replication (and expansion) of Study 1

Taking into account prior knowledge regarding research data handling
Structural model only (without measurement models)

Study 2a

Conceptual Replication (and expansion) of Study 1

Trouble with dwell_time being skewed? -> log-transform?

# Define the SEM model
model <- "
  # MEASUREMENT MODELS
  knowledge       =~ knowledge1 +       
                     knowledge2 +       
                     knowledge3 +       
                     knowledge4
  read_engagement =~ read_engagement1 + 
                     read_engagement2 + 
                     read_engagement3 + 
                     read_engagement4
  understanding   =~ understanding1 +   
                     understanding2 +   
                     understanding3 +   
                     understanding4

  # REGRESSIONS
  dwell_time      ~ knowledge
  read_engagement ~ knowledge
  understanding   ~ knowledge

  # COVARIANCES
  dwell_time ~~ read_engagement
  dwell_time ~~ understanding
  understanding ~~ read_engagement
"
fit <- lavaan::sem(fit, data = data)
summary(fit, standardized = TRUE, fit.measures = TRUE)

Study 2b

Effects of data sharing and data type




Does the information on sharing data influence the participants’ willingness to participate?

(for different data types?)

Study 2b

Effects of data sharing and data type

  • Procedure:
    1. Description & consent of fictitious study presented (see on the right)
    2. then questionnaire

  • Design: 2x3 (within?-)between-subjects design

  • Factors:
    • Data-sharing section (included vs. not included)
    • data collection type (video recording vs. interview vs. survey)

  • Randomized:
    • topic of study (varying degrees of sensitivity)

(Hover over examples to zoom)

Study 2b

Effects of data sharing and data type

  • Measures:
    • Willingness to participate (6-point Likert scale)

  • Treatment checks
    • perceived sensitivity of topic of study
    • Dwell time
    • Self-reported reading engagement
    • Understanding

(Hover over examples to zoom)

Study 2b

Effects of data sharing and data type

Structural model only (without measurement models)
perceived_sensitivity as control

Study 2c




Does the option to clarify concerns about the informed consent make a difference in the willingness to participate?

Study 2c

Effects of feedback on concerns

  • Design: pre-post between-subject RCT
  • Factor:
    • Feedback on concern [yes vs. no]
  • Procedure: extends Study 2b
  • Measures (pre-post)
    • willingness to participate
    • Perceptions of being well-informed about study
    • trust in research?

What does CG do?
Or just measure CG once and paste values to posttest? (-> statistically dependent)

Study 2c

Effects of feedback on concerns

Structural model only (without measurement models)

Questions



General: Idea for a journal? -> registered report?

Study 1: How to ideally collect log-data from past online surveys -> ask in LLiB?

Study 2: How to collect participants? Panel provider probably biased

Study 2: What if participants don’t read the fictitious example? -> ecological validity? internal validity?

Thank you



Jürgen Schneider

References

Brysbaert, M. (2019). How many words do we read per minute? A review and meta-analysis of reading rate. Journal of Memory and Language, 109, 104047. https://doi.org/10.1016/j.jml.2019.104047
Campbell, R., Goodman-Williams, R., Engleton, J., Javorka, M., & Gregory, K. (2023). Open science and data sharing in trauma research: Developing a trauma-informed protocol for archiving sensitive qualitative data. Psychological Trauma: Theory, Research, Practice, and Policy, 15(5), 819–828. https://doi.org/10.1037/tra0001358
Campbell, R., Goodman-Williams, R., Javorka, M., Engleton, J., & Gregory, K. (2023). Understanding Sexual Assault Survivors’ Perspectives on Archiving Qualitative Data: Implications for Feminist Approaches to Open Science. Psychology of Women Quarterly, 47(1), 51–64. https://doi.org/10.1177/03616843221131546
Fichtner, U. A., Horstmeier, L. M., Brühmann, B. A., Watter, M., Binder, H., & Knaus, J. (2023). The role of data sharing in survey dropout: A study among scientists as respondents. Journal of Documentation, 79(4), 864–879. https://doi.org/10.1108/JD-06-2022-0135
Geier, C., Adams, R. B., Mitchell, K. M., & Holtz, B. E. (2021). Informed Consent for Online Research—Is Anybody Reading?: Assessing Comprehension and Individual Differences in Readings of Digital Consent Forms. Journal of Empirical Research on Human Research Ethics, 16(3), 154–164. https://doi.org/10.1177/15562646211020160
Masson, M. E. J. (1983). Conceptual processing of text during skimming and rapid sequential reading. Memory & Cognition, 11(3), 262–274. https://doi.org/10.3758/BF03196973
Parfenova, D., Niftulaeva, A., & Carr, C. T. (2024). Words, words, words: Participants do not read consent forms in communication research. Communication Research Reports, 1–11. https://doi.org/10.1080/08824096.2024.2379832
Pedersen, E. R., Neighbors, C., Tidwell, J., & Lostutter, T. W. (2011). Do Undergraduate Student Research Participants Read Psychological Research Consent Forms? Examining Memory Effects, Condition Effects, and Individual Differences. Ethics & Behavior, 21(4), 332–350. https://doi.org/10.1080/10508422.2011.585601
Steinhardt, I., Fischer, C., Heimstädt, M., Hirsbrunner, S. D., İkiz-Akıncı, D., Kressin, L., Kretzer, S., Möllenkamp, A., Porzelt, M., Rahal, R.-M., Schimmler, S., Wilke, R., & Wünsche, H. (2021). Opening up and Sharing Data from Qualitative Research: A Primer: Results of a workshop run by the research group ,,Digitalization and Science“ at the Weizenbaum Institute in Berlin on January 17, 2020. https://doi.org/10.34669/WI.WS/17
Zuiderwijk, A., Shinde, R., & Jeng, W. (2020). What drives and inhibits researchers to share and use open research data? A systematic literature review to analyze factors influencing open research data adoption. PLOS ONE, 15(9), e0239283. https://doi.org/10.1371/journal.pone.0239283

Credit

Title page: Aziz Acharki on Unsplash

Icons by Font Awesome CC BY 4.0

Slides: link pending